On the value of advanced information about delayed rewards.

In a variety of laboratory preparations, several animal species prefer signaled over unsignaled outcomes. Here we examine whether pigeons prefer options that signal the delay to reward over options that do not and how this preference changes with the ratio of the delays. We offered pigeons repeated choices between two alternatives leading to a short or a long delay to reward. For one alternative (informative), the short and long delays were reliably signaled by different stimuli (e.g., SS for short delays, SL for long delays). For the other (non-informative), the delays were not reliably signaled by the stimuli presented (S1 and S2). Across conditions, we varied the durations of the short and long delays, hence their ratio, while keeping the average delay to reward constant. Pigeons preferred the informative over the non-informative option and this preference became stronger as the ratio of the long to the short delay increased. A modified version of the Δ-Σ hypothesis (González et al., J Exp Anal Behav 113(3):591-608. https://doi.org/10.1002/jeab.595 , 2020a) incorporating a contrast-like process between the immediacies to reward signaled by each stimulus accounted well for our findings. Functionally, we argue that a preference for signaled delays hinges on the potential instrumental advantage typically conveyed by information.

Time, uncertainty, and suboptimal choice.

Under certain conditions, pigeons prefer information about whether food will be forthcoming at the end of an interval to a higher chance of obtaining the food. In the typical protocol, choosing one option (Informative) is followed by one of two 10-s long terminal-link stimuli: SG always ending in food or SR never ending in food, with SG occurring only 20% of the trials. The other option (Non-informative) is also followed by one of two 10-s long terminal-link stimuli: SB or SY, both ending in food 50% of the trials. Although the Informative option yields food with a lower probability than the Non-informative (0.2 vs. 0.5), pigeons prefer it. To determine whether such preference occurs because SG and SR disambiguate the trial outcome immediately upon choice, we delayed the moment the disambiguation took place in two experiments. In Experiment 1, when the Informative option was chosen, SG always ensued for t seconds of the terminal-link, and then the standard contingencies followed. Experiment 2 was similar, except that SR always ensued for t seconds. Across conditions, t varied from 0 to 10 s. In both experiments, preference for the Informative option decreased with t, but the effect was stronger in Experiment 1. We discuss the implication of these findings for functional and mechanistic models of suboptimal choice.

Midsession reversal task with starlings: A quantitative test of the timing hypothesis.

In the Mid-Session Reversal task (MSR), an animal chooses between two options, S1 and S2. Rewards follow S1 but not S2 from trials 1-40, and S2 but not S1 from trials 41-80. With pigeons, the psychometric function relating S1 choice proportion to trial number starts close to 1 and ends close to 0, with indifference (PSE) close to trial 40. Surprisingly, pigeons make anticipatory errors, choosing S2 before trial 41, and perseverative errors, choosing S1 after trial 40. These errors suggest that they use time into the session as the preference reversal cue. We tested this timing hypothesis with 10 Spotless starlings. After learning the MSR task with a T-s Inter-Trial Interval (ITI), they were exposed to either 2 T or T/2 ITIs during testing. Doubling the ITI should shift the psychometric function to the left and halve its PSE, whereas halving the ITI should shift the function to the right and double its PSE. When the starlings received one pellet per reward, the ITI manipulation was effective: The psychometric functions shifted in the direction and by the amount predicted by the timing hypothesis. However, non-temporal cues also influenced choice.

Testing cognitive models of decision-making: selected studies with starlings.

The behavioural sciences are home to controversies that have survived for centuries, notably about the relation between observable behaviour and theoretical constructs addressing out-of-sight processes in the agents' brains. There is no shared definition for cognition, but the very existence of a thriving journal called Animal Cognition proves that such controversies are still live and help to (a) promote research on the complexity of processes leading to action, and (b) nudge scholars to restrict their cognitive models to those that can be falsified experimentally. Here, we illustrate some of these issues in a limited arena, focusing on the construction and expression of subjective value and choice. Using mainly work from our own laboratory, we show that valuation of alternatives is sensitive to options' properties, to subject's state, and to background alternatives. These factors exert their influence at the time the subject learns about individual options, rather than at choice time. We also show that valuation can be experimentally dissociated from the cognitive representation of options' metrics and argue that experimental animals process options independently at the time of choice, without elaborated comparisons along different dimensions. The findings we report are not consistent with the hypothesis that preference is constructed at the time of choice, a prevalent view in human decision-making research. We argue that animal cognition, viewed as a research program at the crossroads of different behavioural sciences rather than as a debate about properties of mental life, is inspiring and solid, and a progressive and progressing paradigm.

Time to contrast models of timing: The structure of temporal memory.

In the study of animal timing over the last 100 years, we identify three different periods, each characterized by a distinct activity. In the first period, researchers brought timing into the laboratory and explored its multiple expressions empirically. In the second period, the growing body of empirical findings inspired researchers to develop a plethora of timing models that vary in theoretical orientation, scope, depth, and quantitative explicitness. We argue that it is now the time to advance towards a third period, wherein researchers select models by comparing them with one another and with data. We make our case by contrasting how the scalar expectancy theory and the learning-to-time model conceive of temporal memory and learning both in concurrent timing tasks and in retrospective timing tasks. We identify four problems related to the structure of temporal memory and to the rules of temporal learning that challenge these models and that should drive the next steps in modeling the timing abilities of animals. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Constantly timing, but not always controlled by time: Evidence from the midsession reversal task.

We used a midsession reversal task to investigate how temporal and situational cues may combine to determine choice in frequently changing environments. Pigeons learned a simultaneous discrimination with 2 stimuli: S1 and S2. Choices of S1 were reinforced only during the first trials, and choices of S2 were reinforced only during the last trials of the session, that is, the reinforcement contingencies reversed once during the session. To weaken the temporal cue (time into the session) that signaled the reversal trial, we varied the location of reversal trial randomly across sessions; to weaken the situational cue (the outcome of the previous trials that might support a win-stay/loose shift strategy), we varied the payoff probabilities associated with S1 and S2. Performance was consistent neither with the exclusive use of a timing strategy, nor with the exclusive use of a situational, win-stay/lose-shift strategy. Instead, choice seemed to be under joint control of both cues. The relative influence of these cues was dynamic: When payoff was higher for S1 than S2, behavior was less time-controlled than when the payoff was higher for S2 that S1, or when they were equal. We advance a descriptive mixture model of joint control for the midsession reversal task. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Base rates bias performance in a temporal bisection task.

We investigated how base rates affect temporal discrimination. In a temporal bisection task, pigeons learned to choose one key after a short sample and another key after a long sample. When presented with a range of intermediate samples they produced a psychometric function characterized by a bias and a scale parameter. When one of the trained samples was more frequent than the other, only the location parameter changed, with the pigeons biasing their choices toward the key associated with the most frequent sample. We then reproduced the bisection task in a long operant chamber, with choice keys far apart, and tracked the pigeons' motion patterns during the sample. Pigeons learned to approach the short key following sample onset, wait on the "short side" for a few seconds, and then, when the sample continued, depart toward the long key. This time-place curve was affected by sample base rate: The probability of pigeons going directly to the long side after sample onset increased when long samples were more frequent than short samples, indicating a decrease of temporal control. We found no evidence of changes in temporal sensitivity. The results are most consistent with models of timing that take into account biasing effects and competition for stimulus control. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The functional equivalence of two variants of the suboptimal choice task: choice proportion and response latency as measures of value.

In the suboptimal-choice task, birds systematically choose the leaner but informative option (suboptimal) over the richer but non-informative option (optimal). The task has two variations. In the standard task, the optimal option includes two terminal link stimuli. In the original task, it includes a single terminal link stimulus. Two models, the temporal information account (Cunningham and Shahan, J Exp Psychol Anim Learn Cogn 44:1-22, 2018) and the ∆-∑ hypothesis (González et al., J Exp Anal Behav 113:591-608, 2020), presuppose that these procedures are equivalent, but no formal comparison is available. Here we test whether or not these procedures are functionally equivalent. One group of pigeons was trained with the standard procedure, another group with the original procedure, and a third group was trained with a hybrid of the other two (i.e., the two options were the optimal links of the standard and original procedures). Our findings indicate that the number of terminal link stimuli in the optimal option is inconsequential vis-à-vis choice. Moreover, our findings also indicate that latencies to respond are a sensitive metric of value and choice. As predicted by the Sequential Choice Model, we were able to predict simultaneous choices from the latencies of sequential choices and observed a substantial shortening of latencies during simultaneous choices.

Step changes in the intertrial interval in the midsession reversal task: Predicting pigeons' performance with the learning-to-time model.

Our goal was to assess the role of timing in pigeons' performance in the midsession reversal task. In discrete-trial sessions, pigeons learned to discriminate between 2 stimuli, S1 and S2. Choices of S1 were reinforced only in the first half of the session and choices of S2 were reinforced only in the second half. Typically, pigeons choose S2 before the contingency reverses (anticipatory errors) and S1 after (perseverative errors), suggesting that they time the interval from the beginning of the session to the contingency reversal. To test this hypothesis, we exposed pigeons to a midsession reversal task and, depending on the group, either increased or decreased the ITI duration. We then contrasted the pigeons' performance with the predictions of the Learning-to-Time (LeT) model: In both conditions, preference was expected to reverse at the same time as in the previous sessions. When the ITI was doubled, pigeons' preference reversal occurred at half the trial number but at the same time as in the previous sessions. When the ITI was halved, pigeons' preference reversal occurred at a later trial but at an earlier time than in the previous sessions. Hence, pigeons' performance was only partially consistent with the predictions of LeT, suggesting that besides timing, other sources of control, such as the outcome of previous trials, seem to influence choice.

Choosing fast and simply: Construction of preferences by starlings through parallel option valuation.

The integration of normative and descriptive analyses of decision processes in humans struggles with the fact that measuring preferences by different procedures yields different rankings and that humans appear irrationally impulsive (namely, show maladaptive preference for immediacy). Failure of procedure invariance has led to the widespread hypothesis that preferences are constructed "on the spot" by cognitive evaluations performed at choice time, implying that choices should take extra time in order to perform the necessary comparisons. We examine this issue in experiments with starlings (Sturnus vulgaris) and show that integrating normative and descriptive arguments is possible and may help reinterpreting human decision results. Our main findings are that (1) ranking alternatives through direct rating (response time) accurately predicts preference in choice, overcoming failures of procedure invariance; (2) preference is not constructed at choice time nor does it involve extra time (we show that the opposite is true); and (3) starlings' choices are not irrationally impulsive but are instead directly interpretable in terms of profitability ranking. Like all nonhuman research, our protocols examine decisions by experience rather than by description, and hence support the conjecture that irrationalities that prevail in research with humans may not be observed in decisions by experience protocols.

Effects of differential probabilities of reinforcement on human timing.

We investigated how differential payoffs affect the temporal discrimination of humans. In a temporal bisection task, participants learned to make one response after a short sample and another after a long sample. When presented with a range of intermediate samples, the proportion of responses fitted well a Gaussian-like distribution function characterized by a location (bias), a scale (sensitivity) parameter, and two asymptote (discrimination) parameters. In Experiment 1, when one response yielded more reinforcers than the other, parameters were unaltered, but overall responses increased for the response producing higher payoffs. In Experiment 2, we used a video game to track motion during the sample and participants learned to approach the "short" response location at sample onset and remain there before departing to the "long" location on long trials. Departure times were shorter when "long" choices produced higher payoffs than "short" and matched well the shifted psychometric functions. However, on some trials, subjects were biased for short, returning to the short side after having departed towards long. Evidence was found for effects of differential payoffs on response bias, but discrimination and sensitivity did not change consistently. These results favor a behavioral account of timing processes.

The Δ-∑ hypothesis: How contrast and reinforcement rate combine to generate suboptimal choice.

When given a choice between two alternatives, each offering food after the same delay with different but signaled probabilities, pigeons often prefer the low probability alternative. This preference is surprising because pigeons fail to maximize the rate of food intake; they exhibit a suboptimal preference. We advance a new explanation, the Δ-∑ hypothesis, in which the difference in probability of reinforcement within terminal links (Δ) and the overall reinforcement probability rate of each alternative (∑) are the key variables responsible for such suboptimal preference. We tested the Δ-∑ hypothesis in two experiments. In Experiment 1, we manipulated the Δs while maintaining constant all other parameters of the task, in particular the ∑s. We predicted a preference for the alternative with the larger Δ. In Experiment 2, we examined the effect of the overall reinforcement probabilities, the ∑s, while maintaining constant all other parameters of the task, in particular the Δs. We predicted a preference for the larger ∑. The results of both experiments support the Δ-∑ hypothesis.

The road ahead for sunk costs.

The sunk cost error is usually ascribed to the "ghost" of past investments, but the "promise-of-a-better-future" account is often difficult to dismiss. The search for the underpinning mechanism(s) is ongoing, but it should go hand-in-hand with a functional account.

Testing the Δ-∑ hypothesis in the suboptimal choice task: Same delta with different probabilities of reinforcement.

In a concurrent-chain procedure, pigeons choose between 2 initial-link stimuli; one is followed by terminal link stimuli that signal reliably whether food will be delivered after a delay; the other is followed by terminal link stimuli that do not signal whether food will be delivered after the delay. Pigeons prefer the former alternative even when it yields a lower overall probability of food. Recently, we proposed the Delta-Sigma (∆-∑) hypothesis to explain the effect: Preference depends on the difference (∆) between the reinforcement probabilities associated with the terminal link stimuli, and the overall probability of reinforcement (∑) associated with the alternative. The hypothesis predicts that, for constant ∑, animals should prefer alternatives with greater ∆ values regardless of the specific probabilities of reinforcement that determine ∆. In 2 experiments, we tested this prediction by comparing a ∆ = .5 against a ∆ = 0 alternative, with the former obtained with different pairs of reinforcement probabilities across conditions. The results supported the hypothesis when the 2 probabilities defining ∆ were significantly greater than 0, but not when one of them was close to 0. The results challenge our theoretical accounts of suboptimal choice and the variables considered to determine pigeons' preference.

The effect of reinforcement probability on time discrimination in the midsession reversal task.

We examined how biasing time perception affects choice in a midsession reversal task. Given a simultaneous discrimination between stimuli S1 and S2, with choices of S1 reinforced during the first, but not the second half of the trials, and choices of S2 reinforced during the second, but not the first half of the trials, pigeons show anticipation errors (premature choices of S2) and perseveration errors (belated choices of S1). This suggests that choice depends on timing processes, on predicting when the contingency reverses based on session duration. We exposed 7 pigeons to a midsession reversal task and manipulated the reinforcement rate on each half of the session. Compared to equal reinforcement rates on both halves of the session, when the reinforcement rate on the first half was lower than on the second half, performance showed more anticipation and less perseveration errors, and when the reinforcement rate on the first half was higher than on the second half, performance showed a remarkable reduction of both types of errors. These results suggest that choice depends on both time into the session and the outcome of previous trials. They also challenge current models of timing to integrate local effects.